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Abstract 

A year-long exploratory project examined how well students 
could effectively respond to a piece of first-year writing using an 
articulated framework—Assignment, Focus, Organization, Support, 
and Proofreading (AFOSP). The students in these small-group 
writing tutorials received peer-facilitated support while they were 
enrolled in first-year composition. Results from a one-way repeated 
measures ANOVA showed statistically significant gains in students' 
abilities to respond to writing using the framework. The findings 
suggest that teaching students to use such a framework can improve 
their ability to critically assess writing. 

T his exploratory study examines students' experiences in small-group, 
credit-bearing writing tutorials in an attempt to identify ways to articulate 
empirical outcomes from the facilitated writing group experience. Like 
many university writing centers, the Writing Center at Washington State 
University (WSU) has struggled to articulate quantifiable outcomes for its 
tutorial programs. The Writing Center views the work done in our small- 
group tutorials as important and necessary, but most of the assessment 
work has been collected qualitatively through student reflective papers in 
narrative form at mid-semester and at the end of the semester. For many 
years, assessment based on students' observations indicating increased 
confidence, acquisition of new skills, and a clearer understanding of effective 
peer reviews seemed sufficient. However, this study came forward as part 
of a desire to use objective measures for improving understanding of what 
students took away from the course. 

Exploring tutorial outcomes through quantitative measures was daunting 
because these researchers had not previously examined what students learn 
from the small-group tutorial experience using empirical methods. Like 
other universities who face the onslaught of accreditation and accountability 
demands handed down from central administration, legislators and the 
public, these researchers were motivated to look for ways to verify students 
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learned over the course of a semester in the small-group writing tutorial. 
Concerns about what might be uncovered from such a study and what such 
implications would hold for the writing program were allayed by the work of 
Haswell (2005), Lerrner (1997, 2003) and Johanek (2000) who advocate for 
the use of empirically based research methods in composition studies. All 
feel such research illuminates knowledge and helps construct our discipline 
in important and crucial ways. 

Kail and Trimbur (1987) provide notable examples of key research into 
Writing Center outcomes when they outlined the theoretical differences 
between the two major types of tutorial offerings. They drew clear distinctions 
between tutorials that operate within the hierarchy of the institution—like 
the Brown Writing Fellows program—(Haring-Smith, 1986) and tutorials 
that operate separately from the curriculum and support student agency 
and development. Grego and Thompson (1996, 2008) have also theorized 
small, group tutorial programs and advocate a separate "ThirdSpace" that 
prioritizes student agency outside of the traditional classroom setting. To 
date, however, there are few examples documenting empirical outcomes 
from such small-group writing tutorials. Kail, Gillespie and Hughes (2010) 
detail outcomes for tutors who have worked in Writing Centers. Diederich 
and Schroeder (2008) document promising outcomes of structured tutorials 
for students who repeated first-year writing and who took a structured 
writing tutorial concurrently. Likewise, High and Damron (2009a, 2009b) 
examine gains made in student's writing and critical thinking abilities in 
engineering courses connected to concurrent, small-group tutorials. None 
of these studies, however, have examined the merits or outcomes from the 
tutorial as an independent entity. This study uses quantitative methods to 
see what students learn from the small-group writing tutorial experience. 

Background 

History and Overview of English 102 

WSU's small-group, credit-bearing tutorial program began in the early 
1990's as part of the WSU Writing Program. The program explored outcomes 
from this freshman-level, small-group, credit-bearing tutorial program, 
English 102. Unlike many universities, the WSU Writing Program is separate 
from the English Department. At WSU, the Writing Program combines 
university-wide instructional support for students through the Writing Center, 
diagnostic assessment of student writing through the Writing Assessment 
Program, and support for faculty through the Writing in the Disciplines 
efforts, and it is housed in the University College. The English Department, 
on the other hand, operates as a traditional academic department—within 
the College of Liberal Arts— that offers courses and degrees. The English 
department is responsible for teaching first-year and other composition, 
creative writing, technical writing, and literature courses. 

While English 102 relies on the first-year writing course to provide students 
with writing assignments with which the group can work in the tutorial, the 
Writing Program has separate aims from the English department and first- 
year writing curriculum. Three purposes are articulated for English 102: 1) 
help students develop the skills needed to effectively recognize and respond 
to strengths and weaknesses in writing, 2) give students the opportunity to 
learn how to give and accept critical feedback on writing, and 3) encourage 


students to participate in an atmosphere of effective peer collaboration. 
The aims of English 102 are complimentary to the first-year composition 
curriculum, but are independent of it. 

The small-group writing tutorials offered at WSU represent a unique 
hybrid of writing center theory. On the one hand, the tutorials exist as 
regular courses within the undergraduate curriculum and are identified to 
complement the work of the first-year writing course. The tutorial sections 
are scheduled with regular course times and meeting places. The small- 
group tutorial provides institutionally required support for writers based on 
a locally developed writing assessment examination. Based on the results of 
the WSU Writing Placement Exam, some students are required to take the 
tutorial, but many students opt to enroll in this course out of a desire for a 
structured tutorial experience. 

On the other hand, the small-group tutorials are facilitated by a peer tutor 
and operate from student-centered writing center practices. Similar to Grego 
and Thompson (1996), our tutorial course was established specifically not 
to be a "hand-maiden" to other disciplines, but rather to exist independent 
of the curriculum and to operate as a ThirdSpace (Grego and Thompson, 
2008). The tutorial operates as a peer-facilitated small-group in which the 
interactions are modeled upon the work of Bruffee (1984) and Kail and 
Trimbur (1987) ushering students into the type of disciplinary conversations 
they will encounter. Unlike the Writing Fellows program (Haring-Smith, 
1986), the tutors are neither agents of the instructors nor the institutional 
hierarchy, but, instead, they are there to help clarify a framework through 
which students can assess the rhetorical aspects of their writing and the 
writing of others (Huot, 2002) going beyond grammatical concerns. 

Toward rhetorical assessment 

The focus of this study is to explore the interaction of the students 
enrolled in the small-group tutorial and the ways they respond to student 
writing to investigate the possibilities of articulating discernable outcomes. 
WSU's English 102 assumes students can be trained to use the vocabulary 
and insight of writing tutors to develop their own writing as well as the writing 
of others by providing feedback through a common framework. Bruffee 
(1984) and Kail and Trimbur (1987) argue for the importance of providing 
instructional settings free from traditionally hierarchal relationships, the 
pressure of instructor expectations, and a focus on grades. Their work argues 
for the importance of peers mentoring other students into the disciplinary 
conversations of the university. Kail and Trimbur assert these spaces needed 
to be separate from the regular curriculum and that a writing center would 
serve as this space. They questioned whether or not such a relationship 
could exist within the curriculum, arguing that a tutor situated within an 
actual writing course only serves as an agent of the instructor (as in the 
Brown Tutor Fellows model) and does not represent a free exchange of ideas 
idealized by Bruffee and peer learning advocates. 

English 102 uses student-centered models to mentor undergraduates 
into the disciplinary conversations with the guidance of a peer tutor (Bruffee, 
1984; Kail and Trimbur, 1987). The relationship is intentionally equal and 
imports the student-to-student interaction into a classroom setting. This 
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study's small-group tutorials allow students to discuss topics and self- 
assess their own writing in a space largely free from the hierarchal systems 
set up in their first-year composition classrooms. The framework of this 
study appears unique; it is rooted in the practices of rhetorical assessment 
articulated by Huot (2002). Huot argues: 

. . . the discourses of assessment, grading, and testing 
have often overemphasized the importance of correctness, 
while at the same time ignoring the importance of rhetorical 
features. Certainly, most writing teachers see the need 
for instruction and emphasis on both grammatical and 
rhetorical aspects of writing. However, what we assess, 
grade, or test ultimately determines what we value. It is 
not surprising, then, that most student revision centers on 
correctness, since the value of correct writing has been 
emphasized over and over again in various assessment, 
testing, and grading contexts. We need to recognize that 
before students can learn to revise rhetorically, they need 
to assess rhetorically (p.170). 

In English 102, tutors are advocates and mentors for the students, not 
"extensions] of the faculty" (Kail and Trimbur, 1987, p. 208). The tutorials 
are not in service to the composition courses, encourage collaboration among 
writers, and give students tools to measure their writing's effectiveness 
independent of their instructors' grades and rubrics. The model disrupts 
traditional power dynamics and beliefs about the ownership and authority 
of knowledge, and "asks students to rely on themselves, to learn on their 
own in the absence of faculty authority figures or their surrogates. They 
must also learn to free themselves from their dependence on the faculty 
continually measuring and certifying their learning" (Kail and Trimbur, p. 
207). 

English 102 provides a regularly scheduled forum for students to meet 
and discuss their writing and is different from the conference/consultation 
model often used by first-year writing instructors or a typical tutorial session 
in a writing center. Kail and Trimbur assert "students have always banded 
together informally, in rap sessions and study groups, to deal with the 
intellectual demands of their experience as undergraduates. Collaborative 
learning, in this respect, is an effort by educators to mobilize the power 
of peer influence toward the intellectual activity of co-learning" (Kail and 
Trimbur, 1987, p. 207). By combining students' natural desire to discuss with 
peers and the formality of a regularly scheduled meeting, English 102 offers 
the philosophy of a writing center tutorial adapted to suit programmatic 
goals of collaborative learning. 

To guide this interaction, WSU's Writing Center developed a framework 
representing a hierarchy of values used to guide tutors in their response 
to student writing (Appendix A). This framework, which has been used for 
nearly twenty years, includes the extent to which the writer attends to 
the course Assignment, provides appropriate Focus, adheres to a relevant 
Organization, integrates Support and evidence into the writing, and utilizes 
Proofreading strategies. The order deliberately prioritizes global issues of the 
writing task over the local issues of grammar and syntax. This framework: 


Assignment, Focus, Organization, Support, and Proofreading (AFOSP) is 
used in the individual peer-to-peer tutorials as well as the small-group 
writing tutorials. 

In an individual tutorial session within the Writing Center, a tutortalks with 
a writer and checks to see if the piece of writing follows the requirements of 
the assignment. Then, as a team, they see if the essay sets a clear context 
for the development of the writer's main point and supporting evidence. In 
addition to context, the "Focus" section concentrates on the writer's thesis 
statement or sentence that drives the main point of the writing. Next, 
students check to see if the writing demonstrates a logical progression of 
ideas, that transitions are present and instrumental to the understanding 
of ideas, and finally that the piece of writing has a recognizable beginning, 
middle, and end. Then they look at the piece of writing to see if the writer's 
assertions are backed up with logical arguments, personal anecdotes, and, 
when appropriate or required, by research which is correctly documented. 
Finally, the tutor and the student examine punctuation, spelling, syntax, and 
other sentence-level issues, and if necessary, patterns of error are identified. 

The AFOSP framework is also used in the small-group writing tutorials, 
but the tutor operates in a slightly different way. Rather than serving as 
the individual consultant on each student's piece of writing, the tutor trains 
all of the students in the small-group tutorial to learn the language of the 
AFOSP framework and then to apply the criteria to other students' writing. 
At the beginning of the semester, the tutor teaches the framework and 
models its use. With each meeting, students take more control of the group 
and interact with each other instead of relying on the tutor to apply the 
framework to their writing. Ideally, as students practice peer review within 
their small-group, they form trust in each other, gain confidence in their 
reviewing skills, ask questions to each other, and rely less on the tutor or 
group facilitator. 

Students should be instrumental in their own growth as writers. Richard 
Straub notes "with remarkable consistency, the recent scholarship on 
response has urged us to reject styles that take control over student texts 
and encouraged us instead to adopt styles that allow students to retain 
greater responsibility over their writing" (1996, p. 223). The approach of 
encouraging student agency ties into both writing center practice as well as 
instructional assessment. Student investment is an essential determinate 
in the work they produce and their likelihood to examine and change their 
own writing process. Student ownership of this process is augmented 
through collaborative learning. "The major characteristics of a studio in 
these different contexts are that learners are producing work, of which they 
take ownership, and that they work both individually and collaboratively 
in some way" (Grego and Thompson, 2008, p. 7). Providing students with 
the language of AFOSP grants them agency to use evaluative criteria in 
examining their own work. Instead of relying on instructors to provide the 
sole or primary feedback for a student's writing, English 102 students are 
asked to evaluate and analyze their own work and the work of their peers. 

Some teachers or administrators might be nervous about the prospect 
of peers taking up the role of providing what Huot (2002) describes as 
"instructive evaluation...[which] is tied to the act of learning a specific task 
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while participating in a particular literacy event (p. 170). Huot argues two key 
features of instructive evaluation exist: 1) assessment is a common feature 
of any literate activity, whether reading or writing, authors and audiences 
are evaluating and categorizing information as they interact with it, and 2) 
the ability to assess quality is an essential component of any interaction (p. 
165). However, Huot avoids conflating grading—as a means of evaluating 
or testing—and assessment—as a strategy to examine the application of 
successful and unsuccessful techniques. "The type of judgment we know as 
grading has little relationship to the type of evaluation writers constantly 
make in the drafting of a particular piece of writing...Giving students an A or 
even a B, even when we suggest revision, probably doesn't encourage them 
to revise, because the grade itself carries more weight as an evaluation than 
what we can say about the need to revise" (p. 167). 

The separation of grading and assessment is what allows students to 
focus on peer feedback rather than a subjective—and final —label of quality 
in English 102. Both peers and tutors are instructed not to discuss grades 
in English 102 and focus only on the writing. English 102 creates an 
environment—separate from the first-year writing course and the instructor— 
where grades and instructor expectations are absent and are replaced by a 
focus on rhetorical assessment and the students' goals for their writing. 

As Lerner (1997, 2003) has noted, not a great deal of empirical 
scholarship is available related to writing center outcomes. Haswell (2005) 
observed—more generally for composition studies—that a study that fails 
to conduct replicable, aggregable, data-driven studies does so at its own 
peril. Other studies for outcomes from structured tutorial experiences show 
promise. This study is guided by the desire to find out if there are discernable 
outcomes that extend the peer-centered philosophy into a course within the 
regular institutional curriculum. 

Description of the study 

Demonstrating how rhetorical assessment can benefit students is only 
half the battle. Without a means to know if or how such a system works, 
this process falls victim to common writing center practice which avoids 
quantifying our work using quantitative measures. Lerner (2003) asserts 
very few published statistical studies of writing center effects exist (p. 61). 
While anecdotal evidence provides comfort that the process has benefits, 
convincing others and ourselves those benefits can be replicated requires 
more data. Traditionally, empirically driven research and Writing Center 
studies have avoided crossing paths. As White (1994) observes: 

The typical evaluation of writing programs (including writing 
projects, writing-across-the-curriculum programs, research 
and grant designs, in-service training seminars, and regular 
instructional programs) usually fails to obtain statistically 
meaningful results. This failure should not be taken to 
mean that writing programs are failures. The inability to get 
results ought, in general, to be seen as a conceptual failure, 
deriving, in part, from a failure to understand the state of 
the art in the measurement of writing ability (p. 248). 


In our Writing Center, there is ample anecdotal evidence students are 
satisfied with their experience in the Writing Center, but it has never been 
systematically or empirically examined whether or not students gain anything 
from their experience in the small-group tutorials. Huot (2002) asserts, 

we need to recognize that before students can learn 
to revise rhetorically, they need to assess rhetorically. 

Certainly much current writing instruction focuses on 
rhetorical concepts, but there is no clear evidence that 
our assessment of student writing focuses on these same 
criteria (p. 170). 

This research project took this type of observation to heart and set out 
to answer whether students learn anything measurable about responding to 
each other's writing in a small-group, credit-bearing writing tutorial using 
the AFOSP framework representing the researchers hierarchy of values. 

Method 

In the spring of 2008, an exploratory research study was set up to 
examine whether or not students were able to learn how to apply the AFOSP 
framework to other students' writing. This framework was developed by the 
WSU Writing Program. This study sought to answer the following questions: 

1. Do students' learn anything measurable about responding to each 
other's writing in a small-group, credit-bearing writing tutorial 
using a course framework that includes the following criteria— 
understanding of Assigned task, Focus, Organization, Support and 
Proofreading (AFOSP) 

2. Is there a difference between the gains made in homogenous group 
tutorials comprised of students from the same section of first- 
year composition compared to mixed group tutorials comprised 
of students from a variety of sections of first-year composition? 

Students enrolled in English 102 during spring semester 2008 were asked 
to participate in this study. There were 411 total participants. The design of 
the study examined students' responses to a piece of writing at the beginning 
of the semester and at the end of the semester, which required looking at 
paired samples. Of the 411 students enrolled in English 102, 72 completed 
the beginning and end of semester responses for this study, which meant 
there were 72 pairs of students' responses to a similar paper. All student 
participants signed an Institutional Review Board approved consent form for 
participation in this study. 

Data Collection 

The study explored two distinct issues: 1) potential outcomes from 
small-group, peer-review sessions and 2) the best method to facilitate this 
peer review. The first phase looked at students' abilities to respond to each 
other's writing using the AFOSP framework to see if gains were achieved 
from the beginning of the semester to the end of the semester; the second 
phase examined whether a difference in gains is present according to the 
make-up of the tutorial (homogenous vs. mixed groups). In other words, the 
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study explored if grouping students in small-group tutorials by similar first- 
year writing course sections was important or if ensuring the enrollments in 
the tutorials had students from various first-year writing courses was more 
important. 

During the first week of tutorials during Spring 2008, the English 102 
tutors were asked to distribute copies of the paper "The Watering Hole" 
(Appendix B). "The Watering Hole" was written for an actual freshman 
composition class by an actual student. The paper, used with permission, 
contains multiple problems. Tutors gave the students a copy of the paper, 
which also briefly outlined the writing assignment. Students were instructed 
to give feedback to help the student revise the paper, and they were asked 
to write their suggestions on the response sheet (Appendix C). Tutors did 
not prompt the type of feedback the students should provide for the paper. 
Students were given ample time during the tutorial to provide written 
feedback for "The Watering Hole." When the students were finished, they 
returned their written responses to the tutor, who subsequently returned 
them to the program administrator. 

Nine weeks later, during the final week of the semester, tutors were 
asked to distribute copies of "The Watering Hole" as well as the response 
sheets. Students were again instructed to write down revision feedback for 
the author of "The Watering Hole." Again, students were not instructed or 
prompted to give any specific type of feedback and they were given ample 
time to write down their revision suggestions. Materials were returned to 
the tutor who then returned them to the program administrator for coding. 
Students responded to the same piece of writing, "The Watering Hole," at 
the beginning and end of the semester to ensure a uniform comparison of 
the quality of their feedback. 

In the first round of student responses to "The Watering Hole," much of the 
initial feedback focused solely on grammar, in some cases providing line-by¬ 
line editorial changes for the sample piece of writing. For example, student A 
gave extensive editorial suggestions: "Take out 'be' in line 1. Change 'that' 
to 'who' in line 2. Change 'Families'" to "Family's." The student focused 
only on grammatical feedback and provided line-by-line editing for two of 
"The Watering Hole's" three pages. While a focus on grammar was common 
for the beginning-of-semester responses, the end of semester responses 
employed more variety in the types of feedback, in some cases writing 
AFOSP along the margin of the page to correspond with their feedback. 
Student A's feedback at the end of the semester still included grammatical 
concerns, but they were not the sole focus. Student A addressed the major 
categories of AFOSP with comments like "The paper seems to lack purpose. 
It fails to answer any of the questions," which directly relates to assignment 
and focus. Student A also asked questions regarding content and purpose, 
such as "how does women socializing at the watering hole tie in?" and "Why 
are you telling us this?" Student A seemed more equipped with a structure 
and vocabulary for responding to writing that went beyond grammatical 
concerns. For a comparison of Student A's response at the beginning and 
end of the semester, please see Appendix D. 

There was an adequate amount of time—at least nine weeks—between 
the first and last data collection points. Students would have been engaged 


in a variety of other activities, homework assignments and extra-curricular 
activities to minimize their recall of "The Watering Hole" thus limiting any 
type of practice effects. Likewise, the tasks for this project at the beginning 
and end of the semester were the same—the students were instructed to 
provide feedback without any particular guidance of the type of feedback 
they should give. Although all of the students had been in the course for 
the entire semester, this study sought to verify empirically that the students 
could learn how to apply the AFOSP framework. For the research project, 
assigning the same task and same sample paper provided appropriate 
comparisons. 

Rating Procedures 

Two separate rating sessions were held to evaluate the quality of 
students' feedback to "The Watering Hole." A group of five Writing Program 
faculty served as evaluators for this project. Raters were asked to evaluate 
each revision suggestion using the criteria of the AFOSP framework. The 
AFOSP Inventory was developed for the raters to collect their evaluations 
(Appendix E). Each revision suggestion was evaluated based on the quality 
of the student's feedback for "The Watering Hole." Raters were instructed 
to evaluate the quality of the student's feedback according to five discrete 
areas: attending to the assignment, focus, organization, support and 
proofreading. Raters were asked to decide if a student's response to a 
particular area represented a weak, adequate or strong response (Table 1). 
Raters were also encouraged to use 'in-between' scores to best reflect their 
assessment of the quality of the feedback. If a student did not provide any 
feedback on a particular area, the rater was asked to mark the box to the 
left of the Weak box to indicate no response. 


Table 1. Example of AFOSP Rating Inventory 


Dimension 

No 

response 

(1) 

Weak (2) 

(3) 

Adequate 

(4) 

(5) 

Strong 

(6) 

Assignment 







Focus 







Organization 







Support 







Proofreading 








As a result, the qualitative assessments of the raters could be translated 
into a six-point scale then used for statistical analysis. This rating methodology 
was adapted from Condon and Kelly-Riley's (2004) methodology for rating 
critical thinking. 

A norming session was held before each rating session to review actual 
samples of students' responses to ensure that raters were employing similar 
understandings of the benchmarks for the rating scale. Likewise, the weak- 
strong benchmarks were emphasized as needing to be defined within the 
context of the type of feedback a freshman could provide. The essays 
were coded so the raters did not know whether the responses had been 
written early or late in the semester. As a result, the 72 student participants 
provided feedback suggestions at the beginning and end of the semester to 
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"The Watering Hole," so a total of 144 student responses were evaluated. 

Stemler's (2004) approach to estimate reliability was followed. Stemler 
argues that "the general practice of describing interrater reliability as a single, 
unified concept is at best imprecise, and at worst potentially misleading." 
Recognizing the limitations in simply examining inter-rater agreement, 
resulted in structuring the study to focus, instead, on how well the raters 
measured the construct of AFOSP and examined the consistency estimates. 
Stemler states "consistency estimates of interrater reliability are based 
upon the assumption that it is not really necessary for two judges to share 
a common meaning of the rating scale, so long as each judge is consistent 
in classifying the phenomenon according to his or her own definition of the 
scale." The first rating session examined the overall quality of students' 
responses on the discrete AFOSP dimensions. This analysis employed a one¬ 
way repeated measures ANOVA. This analysis had a Cronbach's Alpha of 
.7668 indicating very good reliability. The second rating session examined 
the difference in student performance by group composition. This data 
was analyzed using an independent samples t-test. This analysis had a 
Cronbach's Alpha of .7873, again suggesting very good reliability. 

Student grades intentionally were not chosen as a measure for this 
study. While Lerner's studies (1997, 2003) both use grades as a measure 
of Writing Center effectiveness, student grades were avoided as a measure 
for several reasons. First, the study focuses on small-group, peer-facilitated 
writing groups, which have received little to no research attention compared 
to research of Writing Center effectiveness. Actual grades are not assigned 
in the tutorial, as the course is Pass or Fail. Second, while grades may 
be a more commonly appreciated outcome, they are often inconsistent 
across sections of courses. At WSU, over 50 sections of first-year writing are 
scheduled with at least 26 students enrolled in each section. With a program 
this large, variations will exist between student experiences, for peer review, 
instructional feedback, and so on. In addition, the grades in first-year writing 
are reflections of many variables within a course; therefore, extrapolating 
how much the small-group tutorial plays in a student's final grade is difficult. 
Lastly, because our Writing Center has a policy of not discussing grades, our 
choice to avoid them as an outcome is consistent with our pedagogy and 
practice. 


Results 

First, the exploratory study looked at whether or not students improved 
in their ability to respond to each other's writing. A one-way repeated- 
measures, ANOVA, was used to examine 72 students' abilities to apply 
the AFOSP framework to a piece of writing. Students' AFOSP scores were 
averaged together at the beginning of the semester for AFOSP-A and 
again at the end of the semester for AFOSP-B. Intentionally, the analysis 
for the discrete criteria areas of AFOSP was not run in order to see how 
well students used the framework as a whole (as opposed to the individual 
parts). This decision was to prevent any appearance of data mining for the 
best possible outcome, so our analysis relied solely on the use of the entire 
framework. Students' AFOSP scores showed statistically significant gains in 
their abilities to respond to others' writing using the AFOSP framework from 
the beginning of the semester to the end of the semester. 


Table 1: Analysis of Variance for Differences in AFOSP Application 


Source 

ar 

F eta Squared 


Within subjects 


AFOSP Ax 

1 

27.381** .278 

AFOSP B 

71 

(.994) 

A x B within 
group error 




Note. Value enclosed in parentheses represents mean square errors. **p<.01 


A one-way repeated measure ANOVA was run to see if there was a 
difference in students' abilities to apply the AFOSP framework at the 
beginning and end of the semester in a small-group, credit-bearing writing 
tutorial: English 102. An analysis of variance showed that the effect of 
teaching students to apply AFOSP was significant, F(l, 71) = 27.381, p=.05 
such that the beginning AFOSP score was lower (M = 2.26, SD=.92) than the 
end of the semester AFOSP score (M = 3.13, SD = 1.33). The effect size was 
large at .76, and the observed power was .99. The strength of these gains 
was quite strong. 

Second, a separate exploratory study was conducted to see if there 
were differences in students' performances based on the makeup of the 
tutorial group. This study compared homogenous groups, in which students 
from the same sections of the first-year writing course were in the same 
small-group tutorial, to mixed groups in which students in the tutorial group 
were from a variety of first-year composition sections. An independent t-test 
was conducted between the performances of homogenous groups and the 
mixed groups. For the homogenous groups, n = 63 (M = 2.30, SD=.94); 
for the mixed groups, n = 71 (M = 2.59, SD=1.19). No significant difference 
between the tutorial group type and students' ability to apply the AFOSP 
framework F (132)=4.04, p=.120 was found. This meant that the make-up of 
the groups did not affect the students' performances. This was an important 
finding because this information streamlined the WSU registration practices 
which had been very labor intensive. 

A few limitations appeared in this study. First, this study is 
exploratory. Therefore, the researchers caution readers against using these 
findings as strong generalizable claims for in other Writing Center settings. 
The findings from this study for this center were compelling enough to share 
the design and the outcomes for providing an opportunity for others to 
explore the issues in wider writing center contexts. 

Additionally, this is a quasi-experimental study conducted within the 
constraints of an active program. As Campbell and Stanley (1963) observe, 
"there are many natural social settings in which the research person can 
introduce something like experimental design into his scheduling of data 
collection procedures...even though he lacks the full control over the 
scheduling of experimental stimuli" (p. 34). To that end, several limitations 
were found in the interpretation of the findings. First, the results tended 
to be a bit of "a chicken and egg predicament" because true experimental 




34 | TLAR, Volume 16, Number 2 

design dictates a precise and validated definition of the treatment students 
receive as the instructional model for the small-group writing tutorial. 
However, the dynamic and social nature of an actual small-group writing 
tutorial program makes it difficult to extensively chronicle and define the 
instructional treatment. Suffice to say, the AFOSP framework was a common 
tool for all tutors who served as facilitators in the WSU Writing Center and 
it was a reasonable framework to conduct this exploratory investigation. It 
is not, however, feasible to account for the variety of ways in which tutors 
might present the AFOSP framework. Again, as in most Writing Center 
settings, differing approaches are encouraged and it is virtually impossible 
to capture a precise definition of how the AFOSP framework may have been 
presented to the various groups. A dissertation length study could address 
the precision of the facilitation received in small-group writing tutorials. 

Second, limitations exist in the test study population. The WSU Pullman 
campus has a fairly homogeneous population who would enroll in English 
102. Most of the subjects at WSU institution are traditional 18-24- year-olds 
who attend a residential campus. Predominantly, the student population at 
WSU comes from white, middle-class backgrounds. Furthermore, this study 
did not include multilingual writers because most multilingual writers take a 
different first-year writing course than native speakers of English. 

Finally, it was necessary to give students the same piece of writing to 
respond to at the beginning and end of the semester in order to have what 
Campbell and Stanley (1963) call a "time series quasi-experimental design." 
Such a design opens the study up to potential problems with history or the 
interaction of our assessment with other variables since the study takes 
place in a setting that cannot be controlled. But, given the context of the 
exploratory study, having students examine the same piece of writing with 
nine weeks in between was a reasonable way to retain some of the issues of 
experimental control that was forfeited with the dynamic setting of a small- 
group undergraduate writing tutorial. Given these limitations, the findings 
provided hopeful directions that might open up further lines of inquiry. 

Discussion 

The exploratory studies found students in small-group facilitated tutorials 
were able to effectively provide guided feedback for writing improvement. 
According to the analysis, students made statistically significant gains in the 
ability to respond to each others' writing through the AFOSP framework. 
These gains are strong in both effect size and power. Such gains speak to 
Huot's contention that students need to be able to assess rhetorically before 
they can revise their own writing, and given the opportunity, students can 
respond to each other's writing in meaningful ways. Such findings illustrate 
the value of having students work in peer facilitated small-group writing 
tutorials without the pressure of the instructor or grades, and such an 
interaction has meaningful instructional and quantifiable outcomes. 

Likewise, the results also suggest the student make-up of the group 
does not affect their ability to effectively provide feedback to each other. 
For us, this was an important finding. The operating assumption that 
students needed to be in the same tutorial and first-year composition course 
since English 102 was conceived nearly 20 years ago. Trying to make this 


configuration work resulted in huge amounts of administrative effort and 
ran counter to institutional procedures and infrastructure. The findings from 
this study helped the researchers to decide to significantly streamline the 
approach to grouping students. 

This study embraces the concept of necessary peer-facilitated spaces 
that allow students to mentor each other while also supporting a more 
balanced power structure — tutors are student peers. The findings from 
this study suggest a course, facilitated by peer tutors, is possible within 
the curriculum—a separate space, or ThirdSpace in Grego and Thompson's 
terminology—that can produce meaningful student learning outcomes. 

Additionally, this study shows Writing Centeradministratorsand personnel 
empirical studies can be part of writing scholarship. The informal anecdotal 
feedback model has been an intrinsic piece of Writing Center identity and such 
modes of inquiry are central to the way Writing Centers view themselves. 
However, in the modern university's budget climate, Writing Centers may 
exist in more dangerous territory facing cuts or outright elimination because 
of the tendency to stay away from easily quantifiable measures or outcomes. 
Studies like this can be used as a model for other Writing Centers to examine 
and communicate value based on data to university officials. Likewise, such 
studies help us define and articulate outcomes of writing center work in 
ways that Haswell (2005) deems essential to the discipline of composition 
studies. Such empirical forays help validate the theoretical suppositions 
often held about writing center theory and practice. These inquiries help us 
determine the effectiveness of our face-to-face practice with students. 

This study examined students' abilities to provide effective feedback to 
one another based on the rhetorical assessment vocabulary of our hierarchy 
of values framework, AFOSP. The intended outcome of this study was to 
demonstrate teaching first-year writing students a vocabulary based on 
rhetorical assessment, instructive feedback, and writing center pedagogy 
would lead to better evaluation of the work of their peers. While the results 
of this exploratory study support this outcome, the larger implications of 
how these effects translate to a student's success in first-year writing or their 
development as students over their college career remain to be explored. 
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Appendix A 

WSU Writing Program Writing Assessment Criteria 

Assessment Criteria: The following criteria are used to assess both the Writing Placement Exam and the junior 
Writing Portfolio. 

Assignment 

Your assignment attends to the instructions set forth by the instructor. 

Focus 

Your writing demonstrates an understanding of: 

• the assignment 

• essay writing conventions: 

• The essay sets a clear context for the development of your mam point and supporting evidence. 

• The essay uses a vocabulary consistent with your context and which demonstrates an interest in the topic. 

• The essay prorides reader information which is accessible or recognizable to the reader and relevant to your 
mam point. 

Organization 

• Your essay has a recognizable beginning, middle & end. 

• The introduction clearly explams your purpose: the body ‘flows ’ logically m support of that purpose: the 
conclusion prorides a sense of closure. 

• You have chosen the correct format for your writing task. 

• You avoid redundancy or unnecessary repetition of ideas and information. 

Support 

• Your assertions are backed up with personal anecdotes, logical arguments, and. w hen appropriate or required, by 
research which is correctly documented according to MLA (Modem Language Assoc.). APA (American 
Psychological Association), or Chicago (footnoting) documentation styles. 

• Your research can be confirmed by the reader, if necessary. 

Proofreading 

• Punctuation is correct and appropriate. 

• There are no spelling errors. 

• Vocabulary (word choice) is appropriate for a university-level audience. 

• Subjects and predicates are m agreement: tenses of verbs are correct. 

• Pronouns are m agreement with antecedents and consistent with guidelines for non-sexist language. 

• Capitalization is appropriate. 

• Sentence structure is correct (no run-ons. comma splices, fragments). 

• Use of numbers is correct. 
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Appendix B 
“The Watering Hole” 

Used with Permission 

Assignment: Write a 3-4 page profile of a location in Pullman. Use observation and interpretation to give the 
reader some sense of a place. 

Name 

Instructor's Name 
Course Information 
Date 

The Watering Hole 

Since the beginning of time most people have chosen to be wash their cloches every so often, and for those that 
haven't I would rather not discuss them in detail. In the past women have traditionally gone to the local watering hole to 
wash and dry her families' weekly laundry This was also when women often took time to socialize and gossip with other 
women. However, since then the washer and dryer has been invented and people no longer need to do their laundry in a 
nearby stream instead they go to the Laundromat In the twenty-first century usually students, travelers, or the less 
fortunate are the ones who typically use the laundromat. Occasionally die wealthy might be spotted there if die need arises 
to wash then king size comforter (because it won t fit in their brand new Kenmoxe). 

Since I'm a college student and I'm not able to do my laundry at home any longer. I have chosen to go down to the 
local laundromat The idea of this first tune experience excited me I envisioned the typical Laundromat stereotype 
something you would likely see in a movie meeting the love of my life, finding my long lost twin, or maybe even finding a 
winning lottery ncket m they dryer. I doubted I would have such an extreme experience, but was ready for an interesting 
afternoon (plus I would least get to check out cute college guys and. if nothing else, get two weeks of laundry oui of the 
way). 

Before I even reached the laundromat my adventure had already’ begun Reaching my car was a challenge the 
involved struggling with my hands full in trying to open the door and when I looked behind me I realized that my sparkly 
thong was in die middle of the crosswalk, so I sprinted back, extremely embarrassed, and tucked them into my back pocket. 

I then walked a few blocks and entered die stuffy laundromat that smelled of downy fabric softener, and fresh 
Lavender flowers In die midst of die heat I could barely breath, but luckily I held back my sneezes so that I don't drop any 
more thongs, or my hand fill of quarters I took my’ quarters and spent two dollars for two tickets, so I was able to wash and 
dry one load. I threw my clothes along with some soap in the washer, push in the ncket and press start 

I then found a quiet place to sit m the comer and admire the lovely wall decor. The bottom half of the wall is a blue 
chalk color, while the other half is a light blue with big fluffy clouds. These color were probably purchased in a mass 
quantity for a cheap price at a paint store going-out-of-busmess sale. I sit a few feet away from a woman filling out a job 
application, and I look behind me and see a sign saying "Help Warned." I think maybe I could work here, but decide I 
need to get a better feel for the place first. To the other side of me is and a young man doing his mechanical engineering 
homework and sipping from his Starbuck s coffee cup For the rest of the afternoon I sit m my plastic lawn chair typing 
away on my little laptop, constantly peek mg over the top to observe the other laundry doers. I keep thinking to myself 
■where are the cute guys?" Nothing much interesting is happening and I think, "how am I supposed to write about die ugly 
blue walls m possibly* any more detail?" 

The next couple to walk in is carrying large garbage bags of clothing, as if they haven't done their laundry m years 
The woman looks to be no older than twenty-five, and die man she is with looks to be well over fifty. They both have 
wedding rings, so I assume they are married- I sit there staring (like a creep) trying to figure out the relationship of this 
couple I come to the conclusion that he is probably a professor that had an affair with one his young students, got fired, is 
currently unemployed and now living in some cheap college apartment with her. Since he doesn't have a job anymore, he 
has to embarrass himself at the mall laundromat with the wife that is less than half his age The couple then begins to 
separate then laundry by putting them into five different machines I over hear them arguing that they have to spend so 
much money on their laundry they say it is the other partners fault that they have so much laundry. Eventually they settled 
their laundry differences and decided to sit down and read their unusual reading materials 

It perplexes me how any one could possibly study, or read wife all the obnoxious dryer and washer sounds The 
washer cycle begins with a muffled rush of water and then for die next twenrv minutes goes swish, swish over and over. 
The other annoying sounds come from the old creaky* dryer that thud and thump with every rum. I listen and detail these 
awful sounds as I stare at the old Mickey Mouse clock on the childish blue walls. I wait a few more minutes until my 
laundry finally finishes. While transferring my laundry’ from the washer to the dryer. I dread having to stay any longer m 
order to describe how many pounds of din. dust and lmt lie under the machines I think to myself "this might just be the 
most boring expenence of my life, and that listening to my math lectures sounds fun/’ 

I decide to see what else I can find out about this job that is being advertised, so I ask the woman working at the front 
desk what one would do if employed here. She tells me that the job consists of helping customers with laundry problems 
and ironing After she said "ironing" I didn't even listen to the rest of what she had to say. As much as I acknowledge that 
I need to be responsible for my own laundry for the rest of my life. I don't think working m a hot small room ironing other 
people clothes is my* ideal job. no matter what I would be paid. I walk back to my’ old crappy plastic lawn chair and wait 
another twenty minutes, which feels like eternity, until my clothes are finally* dry*. I leave my observation post with a story, 
but not with the nromise of a oavcheck. the love of mv life, mv lost twin, or a winning lotterv ncket. 



Appendix C 
Student Response Sheet 

YOUR TASK: Provide feedback on " The Watering Hole” to help guide the author through revisions 

Include all of your suggestions for revision on this sheet (DO NOT WRITE ON THE WATERING 
HOLE PAPER). Use die back of this sheet if necessary. Your comments do not have to be complete 
sentences. 

WATERING HOLE ASSIGNMENT: Profile a cultural location. 

Recreate a tune and place for the reader usmg specific details and e xamp les Include a strong 
thesis that addresses one or all of the following questions 

1) What does die chosen location say about our culture and'or values? 

2) What informs your reading of this location? 

3) How might people with other perspectives view your location differently from you? 

REVISION FEEDBACK 


Appendix D 

Sample student feedback 

Student A - beginning of semester feedback to "The Watering Hole" 

Past 1 

Take out be'm line 1. 

Change that' to Vho ' m Ime 2. 

Change 'Families'"to "Family':" 

In the past' is past Jc 'have' Jc gone' are present, change to Vent 

"V between stream Jc instead in line 6 

Move ’usually ' m line 6 to between are' Jc 'the'm Hue 7. 

Lowercase laundromat'm line 6. 

Separate if the need arises' with commas 

Lowercase laundromat line 12 

Insert as' between stereotype' 7 'something' line 12. 

Take out 'likely' Ime 12. 
not line 12. 

Insert at between would' Jc 'least' Ime 15 

Change 1 n sentence m line 17 to 'Xp adventure began before 1 reached the laundromat" 

The * to "that' line IS 

Separate with my hands full' with commas Ime IS 
Change'm' to and' Ime 1S 

Take out and' and end sentence with door. 'When starts new sentence line IS 

Page 2 

Change 'them * to 'it' line 2. 

Take out comma after softener' line 4. 

Take out comma after 'things' tine 5. 

Take out took quarters and' Ime 6. take out comma after tickets.' Jc take out 'so I was abte " line 6. 

Change push' to 'pushed' line 7 Jc press ' to pressed ' 

Change 'color ‘ to 'colors' line 10. 

Comma qfler 'quantity ' Jc 'price' line 11. 

Change'sir'm'sat'line 12 
Change 'look ' to looked' line 13. 

Change 'think maybe ' to thought' line 13. 

Change decide ' to decided' line 14. 

Change need 'to needed ' Ime 14. 

'Is'to'was'Ime 14. 

Sit 'to sat 'line 16. 
keep' to kept' line 17. 

"Is' to "was' Jc 'think ' to thought' line 1S. 

"Is ' to "was' line 20 "haven "t" to "hadn 't“ 

Looks' to looked' line 21. 

'Is * to 'was.' looks' to 'looked' line 22. 

Have' to 'had' 'assumed'to assumed' are 'to were ' Ime 22. 

'Sit'to 'sat' come' to came' line 23. 

Student A - end of semester feedback to “The Watering Hole" 

Chose a better 1* sentence that grabs the reader 
How does women sociaitmg at the watering hold hem? 

Thesis? JFJn do we care that people now use the Laundromat' Why are you telling us this? 

A'oid all rhe parenthesis 

You said you carried the stuff to your car. then w alked there... ? 

Too much detail about the wall, off-topic 

Restructure sentences and take out “ “ around your r hough c 

The paper seems to lack purpose. It finis to answer am. of the questions You just told a storyabout your mp to the Laundromat. 
You said how you view the Laundromat, how would other: new it? tQuestion 3) 

There is a lot of irrelevant u\formanon. The first paragraph does not fit with the rest of rhe paper 

Work on conventions Jc grammar spelling Jc typos, same verb tense, tense corfiiston (past Jc present tense, singular Jc plural), 
sentence structure, capitalization, punctuation, convention: 
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Appendix E 

AFOSP Inventory Scoring Sheet 

WSU Writing Program 

Marie the box that indicates the quality of the student's response to ' The Watering Hole" for each dimension: 

WSU ID Number: Color of Check Mark: 

Dimension 


Weak 


Adequate 


Strong 

Assignment 







Focus 







Organization 







Support 







Proofreading 







WSU ID Number: Color of Check Mark: 

Dimension 


Weak 


Adequate 


Strong 

Assignment 







Focus 







Organization 







Support 







Proofreading 







WSU ID Number: Color of Check Mark: 

Dimension 


Weak 


Adequate 


Strong 

Assignment 







Focus 







Organization 







Support 







Proofreading 


























































